Digital watermarking is a process for modifying physical or electronic media to embed a machine-readable code into the media. The media may be modified such that the embedded code is imperceptible or nearly imperceptible to the user, yet may be detected through an automated detection process. Most commonly, digital watermarking is applied to media signals such as image signals, audio signals, and video signals. However, it may also be applied to other types of media objects, including documents (e.g., through line, word or character shifting), software, multi-dimensional graphics models, and surface textures of objects.
Digital watermarking systems typically have two primary components: an encoder (or “embedder”) that embeds the watermark in a host media signal, and a decoder that detects and reads the embedded watermark from a signal suspected of containing a watermark (a suspect signal). The encoder embeds a watermark by subtly altering the host media signal. The physical manifestation of altered host media most commonly takes the form of altered signal values, such as slightly changed pixel values, image luminance, color or color values, picture colors, altered DCT coefficients, etc. The detector component analyzes a suspect signal to detect whether a watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.
Several particular watermarking techniques have been developed. The reader is presumed to be familiar with the literature in this field. Example techniques for embedding and detecting imperceptible watermarks in media signals are detailed in the assignee's U.S. Pat. Nos. 5,862,260, 6,122,403 and 6,614,914, which are hereby incorporated by reference.
Some of the present methods and apparatus are provided to affect embedding a digital watermark (or other steganographic indicia) in signals created by a CMOS or other image sensor. Watermark embedding is preferably integrated “on-chip.” The term on-chip is used to reflect that at least a portion of watermark embedding is facilitated through circuits, electronics or logic provided on silicon or other semiconductor structure associated or in communication with an imaging array. In some implementations the embedding circuitry is co-located with an image sensor array. In other implementations the CMOS sensor array communicates with embedding circuits on a remote chip location.
CMOS imagers are well known in the art, e.g., see U.S. Pat. Nos. 6,645,788, 6,654,057, 6,639,261, 5,708,263 and 5,471,515, which are each herein incorporated by reference. The literature is also full of illustrative work including, e.g., Nixon et al., “256.times.256 CMOS Active Pixel Sensor Camera-on-a-Chip,” IEEE Journal of Solid-State Circuits, Vol. 31(12) pp. 2046-2050, 1996; and Mendis et al, “CMOS Active Pixel Image Sensors,” IEEE Transactions on Electron Devices, Vol. 41(3) pp. 452-453, 1994, which are each herein incorporated by reference.
As discussed in at least some of the above-mentioned patents, a CMOS imager circuit may include a focal plane array of pixel cells, each one of the cells including a photo-detector, e.g., a photogate, photoconductor or a photodiode, overlying a substrate for accumulating photo-generated charge in the underlying portion of the substrate. A readout circuit is connected to pixel cells and may include an output field effect transistor formed in the substrate and a charge transfer section formed on the substrate adjacent the photogate, photoconductor or photodiode having a sensing node, typically a floating diffusion node, connected to the gate of an output transistor. The imager may include an electronic device (such as a transistor) for transferring charge from the underlying portion of the substrate to the floating diffusion node and a device for resetting the node to a predetermined charge level prior to charge transference. In an exemplary CMOS imager, the active elements of a pixel cell preferably perform functions, e.g., of: (1) photon (e.g., light) to charge conversion; (2) accumulation of image charge; (3) transfer of charge to the floating diffusion node accompanied by charge amplification; (4) resetting a floating diffusion node to a known state; (5) selection of a pixel or image area for readout; and (6) output and amplification of a signal representing pixel charge.
A communications channel (or “grid” including rows and columns) overlays most, if not all, of a pixel array to apply timing, control and read out signals and an array of column output signal interconnects. Column lines connect to electronics that are arranged by column outside of the pixel array. This type of architecture allows signals from the entire array, a portion of the array (e.g., rows or pixel windows), or a single pixel in the array to be read out by simple X-Y addressing or row/column clocking.
(Of course, our inventive watermarking techniques apply to other on-chip imagers including CCD sensors and other CMOS configurations. Thus, the above imager description illustrates only a few of the many possible on-chip environments in which our inventive techniques flourish. As will be appreciated by those of ordinary skill in the art, there are many other on-chip environments that are suitably interchangeable with our watermarking methods and apparatus).
A first aspect of the invention is a method of introducing a digital watermark component to imagery generated by an image sensor (e.g., a CMOS or other image sensor). The image sensor includes a plurality of pixels. The method includes providing slight variations to physical characteristics of a set of the plurality of pixels. The variations are provided so as to introduce the digital watermark component during image capture. The method further includes generating imagery with the image sensor. The imagery includes the digital watermark component.
Another aspect of the invention is a method of embedding a digital watermark in image data captured by an imager. The watermark embedding occurs on-chip with the imager, and the imager includes a plurality of pixels. Each of the plurality of pixels is filtered to represent a single color. The method includes, prior to approximating a multi-color value for each pixel, adjusting single color values for select pixels. The adjusting introduces a digital watermark into the image data. The method further includes approximating a multi-color value for each pixel.
GPS or location data can be provided to a watermark embedder located, e.g., on-chip with or external to, an image sensor. The watermark embedder can embed the GPS coordinates or other location data, perhaps with a timestamp, in data captured by the image sensor. Location data can be combined or associated with an operator or photographer identifier (e.g., a plural-bit identifier) to even further identify imagery captured by the image sensor.
Accordingly, yet another aspect of the invention is a method including: i) receiving an image including a digital watermark embedded therein, wherein the digital watermark comprises location information; ii) decoding the digital watermark to obtain the location information; iii) identifying a map depicting a location corresponding to the location information; and iv) providing the map for access over a computer network.
Still another aspect of the invention is a method of visually or graphically identifying locations on a map. The map is to be displayed via a computer display. The method includes receiving data representing a first image, and analyzing the data to obtain information steganographically encoded therein. The information includes or links to first location information associated with the first image. The method further includes providing a map including an area associated with the first location information, wherein the map is to be displayed via a computer display, and identifying a first location on the map that is associated with the first location information. In some implementations identification of a first location includes providing a visual or graphical object on the map at or near a map location that is associated with the first location information. In some cases the object includes a representation (e.g., a “thumbnail”) of the first image.
Another aspect of the invention is a method of representing a geographically location. The method includes receiving an image captured from a first cell phone. The image includes location information steganographic embedded therein. The method further includes decoding the steganographic embedding to obtain the location information; by using the location information or information identified by the location information, identifying a map representing a location associated with the location information; and displaying the map. In some implementations the method includes displaying a visual or graphical object on the map to identify the location. The object may include a representation of the image.
Further features, advantages and aspects of the present invention become even more apparent with reference to the following detailed description and accompanying drawings.